Indonesian Journal of Electrical Engineering and Computer Science 
Vol. 25, No. 1, January 2022, pp. 130~137 
ISSN: 2502-4752, DOI: 10.11591/ijeecs.v25.i1 .pp130-137 o 130 


Segregation of oil palm fruit ripeness using color sensor 


Aiman Mustaffa!, Faiz Arith', Nurin Izzati Fauzi Peong!, Nurul Rafiqah Jaffar!, 


Evelyn Larwy Linggie', Ahmad Nizamuddin Mustafa’, Fara Ashikin Ali? 
'Faculty of Electronic and Computer Engineering, Universiti Teknikal Malaysia Melaka (UTeM), Melaka, Malaysia 
*Faculty of Electrical and Electronic Engineering Technology, Universiti Teknikal Malaysia Melaka (UTeM), Melaka, Malaysia 


Article Info 


ABSTRACT 


Article history: 


Received Jun 10, 2021 
Revised Nov 11, 2021 
Accepted Nov 30, 2021 


Keywords: 


Arduino 

Oil palm fruit 
Ripeness 

Segregation system 
TCS3200 color sensor 


Oil palm is an important industry that has contributed to income and support 
to the economic sector especially for Malaysia and Indonesia. However, 
most of the equipment in the oil palm industry is still operated manually. 
This work developed a system to separate bunches of oil palm fruit using 
color sensors according to maturity level. Fruit color plays a decisive point 
in determining fruit maturity. Here, a specific threshold point of red green 
blue (RGB) was obtained for the determination of the maturity level of oil 
palm fruit. Point values of < 120, 120 < x < 150 and > 150 represent the 
maturity levels of unripe, under ripe and ripe, respectively. This paper is the 
first to report the RGB points for use in the development of automated oil 
palm segregation system in the oil palm plantation industry. Thus, this paper 
will pave the way in producing an accurate and reliable oil palm separation 
system, which in turn has a positive effect in reducing human error. In the 
future, a set of sensors is proposed to detect a bunch of the oil palm fruits. 


This further can speed up the segregation process and more suitable for 
adaptation to the industry. 
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1. INTRODUCTION 

Oil palm plantations are very prominent and play a major role in the economy for countries such as 
Malaysia and Indonesia where oil palm can grow fertile and fast [1], [2]. Most of these palm oil products are 
used for food, detergent and as a biofuel. For the food sector, these crops can be contributing to the food 
sector for the use of cooking oil, shortening and margarine. Oil palm is a productive crop with oil yield per 
hectare per year which is much larger than other oil crops. High oil palm yield means palm oil needs a small 
growth area compared to other competing oil crops and thus become a very attractive source of income. The 
oil composition of the palm kernel oil is used as the main ingredient in the production of non-hydrogenated 
trans-free margarine. Globally, 90% of palm oil is used for food-related product. Furthermore, this palm 
kernel can be used for personal care products in liquid or cream formed [3]. 

In the palm oil industry, Malaysia is the second largest global producer after Indonesia and it has 
been a major economic source for the country for the past few decades. Palm oil cultivation plan was started 
in 1960s to reduce Malaysia’s dependence on rubber and tin under the government’s diversification 
program [3], [4]. Malaysia also became the largest palm oil exporter in the world during the same period. The 
government nationalized three major companies for oil palm which were Guthrie, Golden Hope and Sime 
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Darby in the year 1980 [4]. Based on statistics, up to 2016, the plantation area of palm oil has increased to 
57,400 km2 in all Malaysian states as shown in the Table 1 [4]. 


Table 1. Plan oil plantation area in Malaysia by state in 2016 


State Plantation Area (hectares) _ Percentage of Total Plantation Area Percentage of State Area 
Sabah 1.551,714 27.0% 21.1% 
Sarawak 1,506,769 26.3% 12.1% 
Johor 745,630 13.0% 38.8% 
Pahang 732,052 12.8% 20.3% 
Perak 397,908 6.9% 18.9% 
Negeri Sembilan 178,958 3.1% 26.8% 
Terengganu 171,943 3.0% 13.2% 
Kelantan 155,458 2.7% 10.3% 
Selangor 138,831 24% 17.1% 
Kedah 87,786 1.5% 33.7% 
Melaka 56,149 1.0% 13.5% 
Penang 14,135 0.2% 9.2% 
Perlis 652 0.0% 0.8% 


From Table 1, it can be concluded that in Malaysia, the oil palm fruit produced is in large volume. 
However, oil palm plant companies are still lagging behind in the autonomous system and use manpower in 
most of their production processes [5], [6]. One of them is the palm oil ripeness segregation system. The fully 
ripe fruit consists of high-quality oil compared to the unripe and overripe fruit. In addition, it also contains 
low free fatty acids which healthy foods [7]. Therefore, it is most important to determine that the bunch of 
fruit that are harvested and collected are ripe. The possibility of error in determining the maturity of the fruit 
can affect in higher harvesting costs and substantial losses for the industry [8]. 

At present, there are many automated systems based semiconductor sensor devices have been 
developed [9]-[13]. In agriculture industry, such technology is used to simplify the process for instance, 
nutrient prob [14] loose fruit picking machines [15], [16] and fruit sorting machines [17]-[19]. This 
automated system is powered by a microcontroller and Arduino. Such system can be developed by using the 
electronic components such as a motor (plastic gear motor and direct current motor), color sensor, infrared 
sensor, motor shield and Arduino Mega 2560. 

Many previous studies have been done to develop an autonomous segregation system for the fruits 
and vegetables based on color [1], [20]. Premachandra et. al. [21] developed a pick and place system based 
on color and size using RGB and hue, saturation, value (HSV) values. Color sensor is used to absorb the 
amount light reflected by an object and converted into frequency. The objects are sorted and placed into two 
convenyor assisted with image processing algorithm. However, this system only can perform with duo- 
chrome color objects for the segregation process. Meanwhile, Gaikar et. al. [22] uses audio as the output 
signal. Initially, the sensor detects the color of the object and the RGB serial data is generated and sent to the 
Arduino microcontroller. Then the microcontroller will read the received data and provides the output in 
form OS audio while the LCD will display the type of color. Finally, the audio output is produced through 
the speakers after the voice recorder records and recognizes the color name. Moreover, Raspberry Pi has also 
been used as the control unit for the segregation system with image processing algorithm of Phyton and the 
application of K-means cluster [23]. This system able to segregate the object based on the color, shape and 
size in real time. These features are robust and best to be embedded into the industry. However, system that 
utilize RGB color sensor has not been extensively studied. Although the use of RGB sensors is conventional, 
but a detailed study can provide the desired improvement and its accuracy can be improved. The use of such 
sensors can save a lot of operating and installation costs, and in turn provide added value to the profits of the 
oil palm industry. 

In this regard, this work has developed an automation system that can separate the oil palm fruit 
based on its ripeness. Based on this system, color sensor will detect the color of oil palm fruit and a servo 
motor moves the ripe and unripe fruit to the appropriate production flow. The accuracy of the system will 
also be measured using light intensity feature. 

Normally, 30 months is needed for an oil palm to grow before it can be harvested after being planted 
for the first time. Then, it can bear fruit for the next 20 to 30 years [24]. The oil palm fruit can be categorized 
by 3 types of grades which are unripe, ripen, and overripe as shown in Figure 1. Ripe fruit can be harvested 
from trees using a sickle or machete to process for crude oil production. While the unripe fruit must be left 
until fully ripe. 

Typically, the method used to determine whether a bunch of ripe fruit is by observing the number of 
loose fruits on the ground or the number of loose fruit sockets in the bunch [25]. This method is used for tall 
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trees because it is difficult to determine the ripeness of the fruit by using human vision. Meanwhile, for low 
trees, the harvester can observe the color of the fruit bunches. 

Another concept for determining fruit maturity by looking at the color of the bunch. The most 
important is the color of the fruit because some researchers conclude that the color of oil palm fruit is 
correlated with the content of fruit oil [24]. There are two oil palm genes, named Nigrescens and Virescens 
shown in Figure 2(a) and Figure 2(b), respectively [8]. Both have completely different color changes from 
unripe to overripe. Nigrescens is the most widely grown type in Malaysia where it shows a yellowish-red 
fruit color after ripening. 


pes 
a 


Under ripe Ripe Overripe 


Figure 1. Oil palm fruit ripeness corresponding to its color 


(b) 


Figure 2. The color changes from unripe (left) to overripe (right) for (a) Nigrescens oil palm and 
(b) Virescens oil palm 


Nigerescens have a similar red color before and after ripening which makes the fruit maturity difficult 
to determine with the naked eye [8]. Significant differences can be seen at the ends of the oil palm fruit. The 
color of ripe fruit is dark red at the base while at the tip of the fruit is yellow color. As for the unripe fruit, the 
ends of the fruit are dark purple and yellow at the base. There is a dark red color between purple and yellow. 
The overripe fruit has a darker red color at the base and the yellow color at the tip becomes orange. 

Meanwhile, Virescens has a green-orange fruit color when unripe, the color is bright green—green 
and turns greenish-yellow when ripe. Overall, it will change from green to orange indicating the ripeness of 
the fruit and this difference can be seen obviously and allows the separation process to be perform. 

RGB color is one of the most important criteria for determining the ripeness of fruits or vegetables. 
In the agricultural industry, most of the grading systems used based on the red, green and blue color model to 
perform the segregation process. In general, color can be seen by the human eye but it also depends on the 
light reflected on the surface of the object [25]. 

According to [26], RGB color can be characterized by breaking down the measure of red, green, and 
blue substance of the color apparent as shown in Figure 3. At the point, when an object is brought to the 
detection range of the color sensors, contingent upon their voltage yield of each of the sensor, the color can 
be distinguished [26]. Along with a good lighting and programmed system, the accuracy and precision of 
segregation process will be improved. 


Figure 3. RGB color 
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2. METHODOLOGY 

This work can be categories into two parts namely as hardware and software development. Color 
sensors and the microcontroller were purchased from the market for hardware parts. The latter development 
was generated by the state-of-the-art software design of the Arduino. Both parts were combined and 
embedded to produce a well functional segregation system for this work. 


2.1. Hardware 

In this work, Arduino Mega 2560 microcontroller and programmable color light to frequency 
converter converter (TCS 3200) color sensor were used as the main brain and color detector device 
respectively. The schematic circuit with the outer connection to the hardware is shown in Figure 4. In 
addition, a conveyor system is also developed for the purpose of testing the prototype functionality. The TT 
geared motors generated from dry cell are used to move the conveyor carrying the oil palm fruits. Then the 
moving oil palm fruits will undergo the color determination process for their ripeness. The sensor TCS3200 
which placed vertically on the conveyer will detect the color and provide a set of data in term of light 
intensity to the microcontroller. The Arduino Mega microcontroller will identify the ripeness level of the oil 
palm fruits and produced a serial data of RGB color range. Finally, servo motor that acts as the separator arm 
will drive the oil palm fruit according to its ripeness level into the appropriate container. 

In principle, the TCS3200 color sensor can detect and measure the color range. This sensor consists 
of a sensor chip with 4 LEDs, where it is used to encompass a photodetector array that can detect and 
measure a nearly limitless range of colors. The color sensor detects the color of the oil palm fruits using the 
photodiode and convert into frequency before yields the range of the color. However, in order to determine 
the color accurately, a suitable and close environment is needed. The color sensor is attached vertically facing 
the conveyor inside a medium size box as shown in Figure 5(a). The conveyor will be connected to two TT 
geared motors at each end of the conveyor to move the oil palm fruit. External or interference light exposure 
can be avoided by such close environment. Apart from that, the distance between of the color sensor and the 
oil palm fruit is most important to consider, because there are limitations for the sensor. The distance is set to 
be 15 mm after several optimization processes. 

Figure 5(b) shows the plan view of the prototype where two containers are placed beside and at the end 
of the conveyor. The former container is for ripen fruit and the latter for the unripe fruit. There are a servo motor 
serves as an arm and there are two categories of the output product either ripe or unripe oil palm fruits. 


L293D Motor Driver 
and 74HCS595 Shift 
Register 


Figure 4. Schematic circuit with outer connection to the hardware 
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Figure 5. The prototype of oil palm fruit segregation system from (a) cross sectional view and (b) plan view 


2.2. Software 
This system is using Arduino Mega 2560 as the main controller to develop software design. It is an 
interface for user to create the program for control and support their systems. All the inputs and outputs 
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components are declared in certain code. In this works, the code is create using arduino integrated 
development environment (IDE) software in JAVA language. Figure 6 shows a complete prototype of this 
work. The oil palm fruits are placed at the starting point and move along the conveyor through a color sensor 
box consisting of an IR sensor and a color sensor. The conveyor will stop for three seconds to allow enough 
time for the color sensor to detect the color of the oil palm fruit. After the ripeness is defined, the servo motor 
will move to a position of 0° or 45° to push the fruit that has been identified as ripe or unripe into the proper 
container. 


Figure 6. Prototype of the oil palm fruit segregation system 


3. RESULTS AND DISCUSSION 
3.1. Ripe fruit 

The functionality and accuracy of this system is tested by using several samples that have been 
identified as ripe and unripe product. There are four types of oil palm fruits selected in this experiment. The 
first and second oil palm fruits are ripe as their color have turn into yellow and dark red at the tips and base 
respectively. The developed system has successfully detected and categories them as ripe fruits with RGB 
range value of above 150 indicating a strong signal of ripen as depicted in Table 2 for fruit bunch in 
Figure 7(a) and Figure 7(b), respectively. These experiments have been repeated three times to ensure the 
accuracy of the system is immaculate and also for statistical purpose. 


Table 2. 1“ ripen fruit detected range 


Test R G B Output 
First First 157 189 154 Ripe 
sample 


(a) 
Second 166 184 158 Ripe 
Third 161 178 156 Ripe 
Second First 165 183 151 Ripe 
(a) (b) sample 

: ; ; (b) 
Figure 7. Image for the (a) first sample of ripe fruit and (b) Second 150 172 152 Ripe 
second sample of ripe fruit Tied d6- 1337 162 Ripe 


3.2. Unripe fruit results 

Unripe oil palm fruits as shown in Figure 8(a) and Figure 8(b), the system also successfully 
identified them as unripe products and moved them into the appropriate container. The RGB range values 
indicate that the unripe oil palm fruits produce fewer detection values with most of them are less than 120 
points even after repeated for three times as display in Table 3. Both results of the ripe and unripe oil palm 
fruits have proved that the developed segregation system is functional properly. After repeated the 
measurements for three times, the system shows a consistent result indicating that it is reliable. 

The threshold value obtained is fairly higher than previously reported but shows a similar trend 
where the value increases in parallel from unripe to ripe fruit. Jaffar et. al. [27] and Alfatni et. al. [28] 
reported that the threshold values for R, G and B were 80, 50 and 40, respectively, for ripe fruit bunches. 
Meanwhile, for unripe fruit bunches, the threshold values were slightly lower at 10 points compared than that 
to ripe fruit. However, the values between the two categories are overlapping which indicates that there is 
ambiguity. Recently, RGB image analysis assisted by backscattering imaging demonstrated slightly higher 
average threshold values of 88 and 92 for unripe and ripe fruit bunches, respectively [29]. In addition, a good 
coefficient above 0.8 was recorded proving that its classification is quite accurate and acceptable. Above 
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findings show that the threshold values are varied, depend on the RGB tools and measurement system. 
However, threshold value classification is required for precision to avoid uncertainty. 


Table 3. 1* unripe fruit detected range 


Test R G B Output 
First First 92 92 74  Unripe 
sample 

(a) 
Second 90 90 72  Unripe 
Third 105 102 81  Unripe 
Second First 116 115 95  Unripe 

(a) (b) sample 

(b) 
Figure 8. Image for the (a) first sample for unripe fruit and (b) Second 119 120 102 Unripe 
second sample for unripe fruit Third 105 102 8I _ Unripe 


4. CONCLUSION 

In this work, an automatic segregation system for oil palm fruit using color sensor has been 
successfully developed. The process of segregation is based on the ripeness color of the fruit particularly for 
Nigerescens type. The system has been tested for four samples. This experiment has been performed in 
separating the ripe, under ripe and unripe oil palm fruits quantitatively using RGB range value. This process 
was repeated 3 times, and the results were almost identical, i.e., very encouraging. Based on the RGB data 
series, the ripe oil palm fruit showed a value exceeding 150, while for the unripe fruit showed a value less 
than 120 for all tests. The difference between the two characteristics of the fruit is very significant with a 
difference of the value of 30 and can be categorized as under ripe. This indicates that the probability of an 
error to occur in determining the characteristics of the fruit is slim and thus made this system more reliable 
and robust. In the future, a set of sensors is proposed to detect a bunch of the oil palm fruits. This further can 
speed up the segregation process and more suitable for adaptation to the industry. Furthermore, the image 
processing technology also great to introduce for the segregation system. 
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